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11.5.0 End of Chapter Problems 

Problem 1 

The number of orders arriving at a service facility can be modeled by a Poisson process with 
intensity X = 10 orders per hour. 

a. Find the probability that there are no orders between 10:30 and 11. 

b. Find the probability that there are 3 orders between 10:30 and 11 and 7 orders between 
11:30 and 12. 


Problem 2 

Let {N(t), t G [0, oo)} be a Poisson process with rate X. Find the probability that there are two 
arrivals in (0, 2] or three arrivals in (4, 7]. 


Problem 3 

LetX ~ Poissonifi |) and Y ~ Poisson(ju 2 ) be two independent random variables. Define 
Z = X + Y. Show that 


/ Mi 

X\Z = n ~ Binomial I n, - 

\ Mi+Mi 


Problem 4 

Let N(t) be a Poisson process with rate X. Let 0 < s < t. Show that given N(t) = n, N(s ) is a 
binomial random variable with parameters n and p = ~. 
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Problem 5 

Let /V| (t) and N 2 (t) be two independent Poisson processes with rate A j and X 2 respectively. Let 
N(t) = + N 2 (t) be the merged process. Show that given N(t) = n, 

/ 

;V|(/) ~ Binomial \n, -—— 


Note: We can interpret this result as follows: Any arrival in the merged process belongs to 


/V|(7) with probability 
arrivals. 


^ 1+^2 


and belongs to N 2 (t) with probability 


X\+X 2 


independent of other 


Problem 6 

In this problem, our goal is to complete the proof of the equivalence of the first and the second 
definitions of the Poisson process. More specifically, suppose that the counting process 
{N(t), t G [0, oo)} satisfies all the following conditions: 

1. JV(0) = 0. 

2. N{t) has independent and stationary increments. 

3. We have 

P(N( A) = 0) = 1 - AA + o( A), 

P(N( A) = 1) = AA + o(A), 

P(N( A) > 2) = o(A). 

We would like to show that N(t) ~ Poisson(Xt). To this, for any k G {0, 1, 2, •••}, define the 
function 


gk( 0 = P(N(t) = k). 


a. Show that for any A > 0, we have 


g 0 (t + A) = g 0 (0[l-AA + o(A)]. 


b. Using Part (a), show that 


So(t) 

Soil) 


https://www.probabilitycourse.com/chapter11/11_5_0_end_of_chapter_problems.php 


2/14 



10/8/2018 


End of Chapter Problems 


c. By solving the above differential equation and using the fact that g o (0) = 1, conclude 
that 

g 0 (t) = e~ )J . 

d. For k> 1 , show that 

gk 0 + A ) = gk( 0(1 - AA) + g k _ j(0AA + o(A). 

e. Using the previous part show that 

gk( 0 = - AgiHf) + Ag k - 1(0, 

which is equivalent to 

J t p'grfo] = (')■ 

f. Check that the function 

satisfies the above differential equation for any k> 1. In fact, this is the only solution 
that satisfies g {] (t) = e /J , and g /f (0) = 0 for k> 1. 


Problem 7 


Let {N(t), t £ [0, oo)} be a Poisson process with rate X. Let T x , T 2 , ••• be the arrival times for 
this process. Show that 


fr x ,T 2 , . . . ,T n (h’ h> t n) = e K > for o <t l <t 2 <-< t n . 


Hint: One way to show the above result is to show that for sufficiently small A we have 


< T { < t x + Aj, t 2 < T 2 < t 2 + A 2 ,..., t n < T n < t n + A n j 


X n e | A 2 ---A /? , forO < < t 2 < ••• < t n . 
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Problem 8 

Let {N(t), t G [0, oo)} be a Poisson process with rate A. Show the following: given that N(t) = n 
, the n arrival times have the same joint CDF as the order statistics of n independent 
Uniform (0, t) random variables. To show this you can show that 

n ! 

fr x ,T 2 ,... ,T n \N(t)=n( t l ’ f 2> t n) = ’ for 0 < t x < t 2 < ••• < t n < t. 


Problem 9 

Let {N(t), t G [0, oo)} be a Poisson process with rate A. Let T x , T 2 , ••• be the arrival times for 
this process. Find 


E[T l + T 2 +- + T lQ \N(4)= 10]. 


Hint: Use the result of Problem 8. 


Problem 10 

Two teams A and B play a soccer match. The number of goals scored by Team A is modeled 
by a Poisson process N ft) with rate A j = 0.02 goals per minute, and the number of goals 

scored by Team B is modeled by a Poisson process N 2 (t) with rate A 2 = 0.03 goals per minute. 

The two processes are assumed to be independent. Let N(t) be the total number of goals in the 
game up to and including time t. The game lasts for 90 minutes. 

a. Find the probability that no goals are scored, i.e., the game ends with a 0-0 draw. 

b. Find the probability that at least two goals are scored in the game. 

c. Find the probability of the final score being 

Team A: l, Team B: 2 

d. Find the probability that they draw. 


Problem 11 
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In Problem 10 . find the probability that Team B scores the first goal. That is, find the 
probability that at least one goal is scored in the game and the first goal is scored by Team B. 


Problem 12 

Let {N(t), t G [0, oo)} be a Poisson process with rate 2. Let p : [0, oo) i-» [0, 1] be a function. 
Here we divide N(t) to two processes N x (t) and N 2 (t) in the following way. For each arrival, a 

coin with P(H) = p(t ) is tossed. If the coin lands heads up, the arrival is sent to the first 
process (TV, (7)), otherwise it is sent to the second process. The coin tosses are independent of 

each other and are independent of N(t). Show that N\(t) is a nonhomogeneous Poisson process 
with rate 7(7) = //;(/). 


Problem 13 

Consider the Markov chain with three states S = {1, 2, 3}, that has the state transition diagram 
is shown in Figure 11.31. 



1 

4 

Figure 11.31 - A state transition diagram. 
1 1 

Suppose P(Xy = 1) = - andi^Xj = 2) = 

a. Find the state transition matrix for this chain. 

b. Find P(X { = 3, X 2 = 2,X 3 = 1). 

c. Find P(X x = 3,X 3 = 1). 
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Problem 14 

Let « 0 , «j, ••• be a sequence of nonnegative numbers such that 

00 

X a j =L 

7=0 

Consider a Markov chain X Q , X^X 2 , ••• with the state space S = {0, 1, 2, such that 

p n = a.:, for all j G S. 

y j 

Show thatXj, X 2 , ••• is a sequence of i.i.d random variables. 


Problem 15 

et X be a discrete-time Markov chain. Remember that, by definition, 

( f'l') 

p -j = P(X n = i \Xq = /'). Show that state i is recurrent if and only if 

00 

Z (n) 

Pii = G0 - 

n = 1 


Problem 16 

Consider the Markov chain in Figure 11.32. There are two recurrent classes, R j = {1,2}, and 
R o = {5, 6, 7}. Assuming X Q = 4, find the probability that the chain gets absorbed to R { . 
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Problem 17 

Consider the Markov chain of Problem 16 . Again assume X 0 = 4. We would like to find the 
expected time (number of steps) until the chain gets absorbed in R \ or R 2 . More specifically, 
let The the absorption time, i.e., the first time the chain visits a state in or R^. We would 
like to find E[T\X Q = 4]. 


Problem 18 

Consider the Markov chain shown in Figure 11.33. Assume X Q = 2, and let Abe the first time 
that the chain returns to state 2, i.e., 

N= min {n > 1: X n = 2}. 


Find E[N\X 0 = 2], 
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j. 



Figure 11.33 - A state transition diagram. 


Problem 19 

Consider the Markov chain shown in Figure 11.34. 



Figure 11.34 - A state transition diagram. 

a. Is this chain irreducible? 

b. Is this chain aperiodic? 

c. Find the stationary distribution for this chain. 

d. Is the stationary distribution a limiting distribution for the chain? 


Problem 20 

(Random Walk) Consider the Markov chain shown in Figure 11.35. 
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Figure 11.35 - Simple random walk. 

This is known as the simple random walk. Show that 

=( n n y^-pT< 

o- 

Note: Using Stirling's fonnula, it can be shown that 

00 00 

k =l «=i\» / 

l 

is finite if and only if p ± -. Thus, we conclude that the simple random walk is recurrent if 

1 1 
P = 2 anc * is transient if/; d ~ (see Problem F5 ). 


Problem 21 

Consider the Markov chain shown in Figure 11.36. Assume that 0 < p < q. Does this chain 
have a limiting distribution? For all i,j G {0, 1, 2, •••}, find 

hm P(X n =j\X Q = i). 

n —> go 



Q q q 


Figure 11.36 - A state transition diagram. 


Problem 22 

Consider the Markov chain shown in Figure 11.37. Assume that p >q> 0. Does this chain 
have a limiting distribution? For all i,j E {0, 1, 2, •••}, find 

hm P(X n =j\X Q = i). 
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q g q 


Figure 11.37 - A state transition diagram. 


Problem 23 

(Gambler's Ruin Problem) Two gamblers, call them Gambler A and Gambler B, play 
repeatedly. In each round, A wins 1 dollar with probability p or loses 1 dollar with probability 
q = 1 - p (thus, equivalently, in each round B wins 1 dollar with probability q = 1 - p and 
loses 1 dollar with probability p). We assume different rounds are independent. Suppose that 
initially A has i dollars and B has A - i dollars. The game ends when one of the gamblers runs 
out of money (in which case the other gambler will have N dollars). Our goal is to find p-, the 

probability that A wins the game given that he has initially i dollars. 

a. Define a Markov chain as follows: The chain is in state i if the Gambler A has i dollars. 
Here, the state space is S = {0, 1, •••, N}. Draw the state transition diagram of this chain. 

b. Let a - l be the probability of absorption to state N (the probability that A wins) given that 

Xq = i. Show that 


a 0 = 0, 
= lj 


q 


a i+ \ ~ a i 


for i = 1, 2, •••, N~ 1. 


c. Show that 


a i = 


q (q \ 2 /^V _1 

p\p \p 


a j, for i = 1, 2, •••, N. 


1 1 

d. Find a t for any i G {0, 1, 2, •••, N}. Consider two cases: p = ^ and p ± y. 


Problem 24 

Let N = 4 and / = 2 in the gambler's ruin problem ( Problem 23) . Find the expected number of 
rounds the gamblers play until one of them wins the game. 


https://www.probabilitycourse.com/chapter11/11_5_0_end_of_chapter_problems.php 


10/14 



10/8/2018 


End of Chapter Problems 


Problem 25 

The Poisson process is a continuous-time Markov chain. Specifically, let N(t) be a Poisson 
process with rate A. 

a. Draw the state transition diagram of the corresponding jump chain. 

b. What are the rates A • for this chain? 


Problem 26 

Consider a continuous-time Markov chain X{t) that has the jump chain shown in Figure 11.38. 
Assume A j = A 2 = A 3 , andA 4 = 2Aj. 



Figure 11.38 - The jump chain for the Markov chain of Problem 26. 

a. Find the stationary distribution of the jump chain n = \n 1? tt 2 , 7t 3 , 7t 4 ]. 

b. Using k, find the stationary distribution for X(t). 


Problem 27 

Consider a continuous-time Markov chain X(t) that has the jump chain shown in Figure 11.39. 
Assume A j = 1, A 2 = 2, and A 3 = 4. 

a. Find the generator matrix for this chain. 
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b. Find the limiting distribution forX(t) by solving nG = 0. 

1 



Figure 11.39 - The jump chain for the Markov chain of Problem 27. 


Problem 28 

Consider the queuing system of Problem 3 in the Solved Problems Section ( Section 3.4 ). 
Specifically, in that problem we found the following generator matrix and transition rate 
diagram: 


T~X X 0 0 ••• 1 


G = 


jx ~ (ju + A) X 0 
0 /j ~(ju + X) X 


The transition rate diagram is shown in Figure 11.40 

AAA 



M M M 


Figure 11.40 - The transition rate diagram for the above queuing system. 
Assume that 0 < X < ju. Find the stationary distribution for this queueing system. 
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Problem 29 


Let W(t) be the standard Brownian motion. 

a. Find P( - l < W(l) < 1). 

b. Find P(1 < W( 2) + W( 3) < 2). 

c. Find P(W{\) > 2 | W(2) = 1). 


Problem 30 

Let W(t) be a standard Brownian motion. Find 

P (0 < W(l) + W(2) < 2, 3W(1) ~ 2W(2) > 0). 


Problem 31 

(Brownian Bridge) Let W(t) be a standard Brownian motion. Define 

X{t) = W(t ) - tW( 1), for all t G [0, oo). 
Note thatX(0) = X(l) = 0. Find Cov(X(s), for 0 < s < t < 1. 


Problem 32 

(Correlated Brownian Motions) Let W(t) and U(t) be two independent standard Brownian 
motions. Let -1 < p < 1. Define the random process X{t) as 

X{t) = pW(t ) + ^1 - p 2 U(t), for all t G [0, oo). 


a. Show that X(t) is a standard Brownian motion. 

b. Find the covariance and correlation coefficient ofX(t) and W(t). That is, find 
Cov(X(0, W(t)) and p(X(t), W(t)). 


Problem 33 
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(Hitting Times for Brownian Motion) Let W(t) be a standard Brownian motion. Let a > 0. 
Define T a as the first time that W(t) = a. That is 

T a = min {t: W{t) = a). 

a. Show that for any t > 0, we have 

P(W(t) >a) = P(W(t) >a\T a < t)P{T a < t). 

b. Using Part (a), show that 


P(T <0 = 2 



c. Using Part (b), show that the PDF of T a is given by 


a | <r 

f T (0 = ^^ ex P {~ 

a ty27Tt I 2t 


Note: By symmetry of Brownian motion, we conclude that for any a ^ 0, we have 


| a | | a 

f T (0 = H= ex P 
t\\2nt 


21 


previous 
next —> 
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